DOCUKENT RESUME 

ED 321 687 HE 023 705 



AUTHOR 
TITLE 



PUB DATE 
NOTE 



PUB TYPE 



EDRS PRICE 
DESCRIPTORS 



IDENTIFIERS 



Haberaec)cer, Heather J. 

Developing InterinstAtutional Comparisons of 
Nonacademic Units for Use in the Review Process. AIR 
1990 Annual Forum Paper. 
16 Nay 90 

36p.; Paper presented at the Annual Forum of the 
Association for Institutional Research (30th, 
Louisville, KY, May 13-16, 1990). 
Guides - Non-Classroom Use (055) — 
Speeches/Conference Papers (150) 

MF0l/l'C02 Plus Postage. 

College Programs; Colleges; Comparative Analysis; 
Higher Education; Information Sources; Institutional 
Characteristics; Literature Reviews; Models; 
^Organizational Effectiveness; ^Program 
Effectiveness; ^Program Evaluation; *Units of Study; 
Unive^ sities 

*Air ?orum; Northeastern Illinois University; 
*Program Review 



ABSTRACT 

A frameworJc is presented for developing and utilizing 
interinstitutional comparisons of nonacademic units in the program 
review process using Northeastern Illinois University of Chicago, 
Illinois, as an example. The frameworic is drawn from a review of the 
literature on organizational effectiveness and the various dimensions 
of management indicators. Issues are discussed regarding how to 
establish appropriate comparative institutions, determine the 
appropriate level and unit of analysis, and identify available 
sources of information. Additionally, various reans of interpreting 
the information derived from the process art explored as well as some 
of the pitfalls to avoid in the evaluation process. Contains 21 
references . (Author/GLR) 



* Reproductions supplied by EDRS are the best that can be made 

* from the original document. 



00 
CO 
IH 

CO 
Q 



Developing Interinstitutional Comparisons of 
Nonacademic Units for Use in the Review Process 



Presented by: 

Heather J. Haberaeclcer 
Assistant Vice President for Budget and Finance 
Northeastern iiiinois University 
5500 N. St. Louis 
Chicago, iL 60625 



BEST COPY AVAILABLE 



m 



U.S DEPARTMENT or EDUCATtON 
Ofttcr of EdtJCCtKjnai RMMfCh and improv«mant 

EDUCATIONAL RESOURCC5 INFORMATION 
CENTER (ERiC) 

t^Thia documani h«a b«an raproducad •• 
r«caiv«d from th« p«rton or organitation 
«Xi0irt«ting it 

□ MirH>r Chtng*! hava twan mtd« to improv« 
i«prOductton quality 

• PointaotvMworoptnioniatatftdinthiadocu- 
m#nt <tr not n#c*Manly rtprtMnt official 
OfcRl position or podcy 



1990 AiR Forum 



Me/ 16, 1990 



2 



"PERMISSION I O REPRODUCE THIS 
MATERIAL HAS BEEN GRANTED BY 



AIR 



TO THE EDUCATIONAL RESOURCES 
INFORMATION CENTER (ERIC)." 



fof Mmmg$m$nt Rmmrch, MIcy Amiytis, and Planning 



This paper was presented at the Thirtieth Annual 
Forum of the Association for Institutional Research 
held at The Gait House, Louisville, Kentucky, 
May 13-16, 1990. This paper was reviewed by the 
AIR Forum Publications Committee and was judged to 
be of high quality and of interest to others 
concerned \rf.th the research of higher education. 
It has therefore been selected to be included in the 
ERIC Collection of Forum Papers. 

Jean Endo 
Chair and Editor 
Forum Publications Editorial 
Advisory Committee 



ERLC 



3 



Abstract 



The extent to which the productivity or efficiency of units can or should be evaluated in 
the context of a nonacademic program review has been the subject of some disagreement 
and debate. This paper presents a framework for developing and utilizing interinstitutional 
comparisons of nonacademic units in the program review process. Issues regarding how 
to establish appropriate comparative institutions, determine the appropriate level and unit 
of analysis, and identify available sources of information will be discussed. Various means 
of interpreting the information derived from the process will be explored as well as some 
of the pitfalls to avoid in the evaluation process. 
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Introduction 

In the early years of program review, attention was focused primarily on the review 
of academic programs. Program review results were seen as a necessary vehicle for more 
informed decision-making and as a means of assessing the appropriateness, adequacy, 
effectiveness and efficiency of academic programs. By 1982, a national study showed that 
over 82 percent of the institutions surveyed had some type of formal evaluation process 
in place for academic progran.s (Barak, 1982). 

More recently, faculty, administrators, and, in some cases, state higher education 
officials have begun to focus on the magnitude of institutions' investment in nonacademic 
programs designed to support instructional, research and public service programs. In some 
cases, this investment can equal 40 percent of an institutions unrestricted fund budget. 
This substantial investment combined with either the actual or perceived increase in the 
size of administrative operations has led to an increasing number of questions being raised 
about the effectiveness and efficiency of these nonacademic units. 

While the number of questions being raised abouf the effectiveness and efficiency 
of nonacademic units may be increasing, it does not appear that the number of institutions 
adopting a formal and systematic process for the review of nonacademic units has 
increased substantially in response to these questions. For example, a review of the 
literature shows that the University of Illinois (Wilson, 1987) and the University of 
Maryland - College Park [UMCPJ (Brown, 1989) are among only a handful of institutions 
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which have repcrtea the development of ongoing processes for the review of nonacademic 
units. 

If one assumes that most higher education institutions face resource constraints at 
some level and that nonacademic units can account for up to 40 percent of an institution's 
budget from unrestricted funds, then why are not more nonacademic program review 
processes being developed? Wilson (1987) provides at least some explanation, noting the 
difficulties involved in designing unique evaluation plans for each separate unit. 

However, are such reasons merely a smoke screen for more fundamental problems 
causing higher education administrators to shy away from the development of nonacademic 
program review processes? For example, in spite of the stated need to develop unique 
evaluation plans for each nonacademic unit, both the University of Illinois and the 
University of Maryland - College Park have developed a core set of questions designed to 
evaluate the operation of a unit. User surveys serve as one means of addressing the 
effectiveness dimension of units under review. Regardl'*^ of whether academic or 
nonacademic units are being reviewed, processes can be designed which meet the three 
characteristics of an effective program review suggested by Ewell (1983), that is, that they 
be ^tematic, regular, and comprehensive. Therefore, many of the concerns being raised 
about the difficulties involved in designing nonacademic program review processes can be 
addressed. 



The o dimension of effectiveness which is clearly missing from the frameworks 
developed to date for the review of nonacademic units is an evaluation of the efficiency 
or productivity of the unit. Brown (1989) notes this omission as one of the main 
shortcomings of the UMCP model. Others (Wilson, 1987) discuss the difficulty of 
collecting comparable data in this area. It is this researcher's contention that until such 
time that explicit measures of efficiency or productivity of nonacademic units are built into 
the nonacademic program review process and the methodologies behind such measures 
receive as widespread acceptance as unit cost and faculty load data for academic prograr 
few institutions will be willing to expend the resources required to mount an ongoing 
process for the renew of nonacademic units. 

One means of developing explicit measures of efficiency or productivity of 
nonacademic units is through the use of comparative data. This paper presents a 
framework for developing and utilizing interinstitutional comparisons for nonacademic units. 
The framework is drawn from a review of the literature on organizational effectiveness and 
the various dimensions of management indicators. 

Literature Review 

A review of the literature shows that the subject of productivity or efficiency is 
clearly detailed in some models of organizational e^ectiveness while totalty missing in 
others. It appears that the ^ubject is most consistently addressed in the systems model of 



organizational effectiveness whereby organizations arc viewed as open systems interacting 
With the environment. Salancik and Pfeffer (1974) discuss organizations* dependency on 
a cydt of resource acquisition, throughput and output for their very survival Yuchtman 
and Seashore (1967) also found resource acquisition to be so fundamental that they have 
suggested it as a measure of organizational effectiveness. 

The resource dependency dimension of organizational effectiveness was noted early 
on in the definition of organization effectiveness offered by Georgopoulos and 
Tannenbavm (1957). They defined organizational effectiveness as "the extent to which an 
organization as a social system, given certain resources and means, fulfills its objectives 
without incapacitating its means and resources and without placing undue strain upon its 
members" (Georgopoulos and Tannenbaum, 1957:535). Their concept of organizational 
effectiveness included 1) organizational productivity, 2) organizational flexibility to adapt 
to both internally and externally induced organizational change and 3) the absence of 
interorganizational tension or control as among the salient criteria. 

Katz and Kahn (1978) discuss effectiveness in terms of efficiency and poh'tical 
effectiveness. While some authors appear to use the terms efficiency and productivity 
interchangeably, others are emphatic in noting that the concepts are not equivalent. 
Lindsay (1982), for example, defines efficiency in terms of the extent to which the level of 
input is optimal relative to the level of output. Productivity, on the other hand, is defined 
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as the ratio of output to input measured in physical or monetary teims as adjusted to be 
independent of any change in price (Lindsay, 1982: 179). 

While the subject of efficiency or productivity is frequently cited as one dimension 
of effectiv ness in the systems model, in the goal attainment model of organizational 
effectiveness, e^ciency and effectiveness are often viewed as independent concepts. 
Advocates of the goal attainment model generally agree that effectiveness refers to goal 
attairment while efficiency refers to the costs incurred in goal attainment. The 
measurement of effectiveness in this model is viewed independent of the level of resources 
committed to tht organization. In criticizing the model, Hannan and Freeman (1977) raise 
the issue of whether an analysis of goal attainment without considering the level of 
resources provided has much meaning or whether any evaluation of organizational 
performance has at least implicit cost comparisons built into the framework. Given the 
relationship between resources and performance, they conclude that when resources are 
scarce, performance evaluations that do not take cost into consideration are not very useful 
(Hannan and Freeman, 1977:110). 

In summary, treatment of the issues of resource dependency and efficiency are key 
distinctions between the goals attainment model and the systems model of organizational 
effectiveness. Literature in the field of management appears to be developing as a means 
of bridging the gap between resource dependency/efficiency and goals attainment in the 
evaluation of organizations. Elkin and Molitor (1985) advocate the use of management 
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indicators as a means of monitoring whether resources are utilized in an effective, efficient 
and appropriate manner towards meeting the goals of the organization. Among the salient 
features which define a management indicator i^tt tne fact that: 1) it addresses one of a 
limited number of areas designated by top management as critical to the continuing 
successful functioning of the organization, 2) it is made up of a ratio of operating statistics, 
3) there exists a criterion or standard against which the actual indicator value can be 
compared and 4) it is analyzed as a trend over time, preferably over four or more time 
periods (Ellrin and Molitor, 1985:14). 

The conceptual framework developed by Elkin and Molitor for selecting 
management indicators contains dimensions of effectiveness, efficiency and adequacy of 
finances (See Figure 1). Of particular relevance to this discussion is their definition of the 
efGciency dimension. In describing the relationship between inputs and outputs, Elkin 
and Molitor note that a measure of efficiency is relative; it is only useful when compared 
to a standard or another measure of efficiency which provides a base line of experience 
(1985:19). They also note that an increase in efficiency can occur either as a result of 
fewer resource inputs being required to produce a given amount of output or as a result 
of a greater amount of output being produced for a given amount of resources. This 
distinction will become more relevant when applied to the interpretation of results from 
interinstitutional comparisons. 
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FIGURE 1 

CONCEPTUAL FRAMEWORK f'OR SELECTING 
MANAQEMENT INDICATORS IN NONPROFIT ORGANIZATIONS 
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Two dimensions of efficiency highlighted in the framework are capacity 
utilization/productivity and unit costs. Capaoi^ utilization is an assessment of the efficient 
use of available resources and is dependent upon a known, finite capacity (Elkin and 
Molitor, 1985:19). Given this limitation, the capacity utilization dimension has applicability 
to only a limited number of nonacademic units such as a counseling center whereby the 
number of available counseling sessions per counselor can be multiplied by the total 
number of counselors and compared to counseling sessions utilized. On the other hand, 
staff productivity, an efficiency measure of an organization's personnel, can have general 
applicability to the evaluation of nonacademic units particularly when measured on a 
comparative basis. 

The other efficiency dimension, unit costs, describes the ratio of outputs to inputs 
as measured in financial tern::. Again, when measured in relative terms, unit costs can 
have general applicability to the evaluation of nonacademic units. 

A review ^f the literature suggests that it would be appropriate to evaluate the 
relative efficiency of a nonacademic unit as one dimension of an overall evaluation of the 
effectiveness of the unit. Various means of measuring the relative efficiency of units, 
including the use of management indicators, are addressed. In the next section, the 
framework selected for use on the researcher's home campus for developing and utilizing 
comparative data in the ev'^luation of nonacademic units is presented. 
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Framnwork for Comparison 

In this section, the framework developed for evaluating the re ative efficiency of 
nonacademic units at Northeastern Illinois University will be discussed. Northeastern 
Ulinois University is a comprehensive public university of 10000 students in Chicago which 
was founded as a city-owned teachers college in 1961. It should be noted that the relative 
efficiency measuu^ developed were only one component of a more comprehensive 
nonacademic program review process instituted in 1982. Like the University of Illinois, 
much of the impetus for developing a nonacademic program review process at 
Northeastern came from a request from the Illinois Board of Higher Education to provide 
them with program review results on each nonacademic and administrative unit every five 
years. 

The various means of determining comparative institutions for this purpose will be 
discussed first The methodology emplc^ed in developing comparable nonacademic units 
will be addressed next along with the role of unit managers in identifying such units. 
Particular attention will be given to the unit of analyses and level of analysis used in the 
comparisons as these issues have proven to be critical to the acceptance of the model by 
various administrators and unit mana^rs on the researcher's home campus. The utility 
of various sources Oi information available for comparison will also be discussed. Finally, 
some pitfalls to avoid in developing and interpreting comparative data will be suggested. 
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Coaparativs institutions. Over the years, various statistical techniques 
such as cluster analysis, factor analysis and discriminant analysis and other less statistically- 
oriented means have been employed to develop peer institutions for comparative purposes 
(Brinkman and Teeter, 1987). While differences exist in the methodologies used, among 
the types of variables usually considered in developing peer groups are the following: 
enrollment, number of degrees earned, programs offered including level, professional 
staffing and average salaries (Teeter and Christal, 1985), 

Given that the emphasis of this comparative effort is on nonacademic and 
administrative units, it is particularly important that enrollment level be a key variable in 
whatever means is used for selecting a peer group of institutions. The need for this 
emphasis comes from the resvlts of research on economies of scale in higher education 
which shows that the administrative area typically experiences the greatest reduction in unit 
cost as enrollment increases (Brinkman and Leslie, 1985). 

The methodology used in this study to select peer ins^litutions for evaluating the 
relative efficiency of nonacademic units was developed by the Illinois Board of Higher 
Education to aid it in the policy-making process (Illinois Board of Higher Education, 1985). 
Since this methodology already had credibility with the body to which the review results 
were to be submitted as well as with many institutional representatives, it was a logical 
choice. 
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For Northeastern Illinois University, % peer institutions from across the country 
were selected. Of this number, one other Illinois public university was reported in the 
same peer group. In addition, three other Illinois public universities were reported in the 
peer group most similar to Northeastem's peer g^oup. Since two of these three institutions 
were governed by the same governing board as Northeastern, it was i cided to include 
them as comparative institutions. The remaining institution from this peer group was 
invited to participate in the study bat declined. A decision was made early on to include 
only Illinois public universities in the initial study in order to avoid opportunities for error 
in interpreting data from institutions unfamiliar to the author. Therefore, a total of four 
institutions, including Northeastern, provided comparative data on nonacademic units. 

Conpmrmtiv^ NonacadMio Units. The internal budget books of the other 
three institutions were first reviewed to determine the extent to which units similar to those 
at Northeastern existed at the other campuses. Among the types of information provided 
in the budget books for the current year were title and account number of the unit, title 
and position number of each position budgeted, the number of months budgeted for each 
position, the total amount budgeted for each position, and the total amount budgeted for 
nor^personnel costs by line item. 

A unit by unit comparison was made between departments at Northeastern and 
those at the other institutions o determine differences in organizational structure between 
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the institutions. A review of position titles proved particularly helpful in identifying where 
functions housed in one department at Northeastern are housed in a different department 
at the other institutions. For example, such a review showed that the student employment 
function is housed in the Placement Office at Northeastern and in the Financial Aid Office 
at all of the other institutions. In cases where either the unit or position title provided no 
indication of the function of the unit or position, the budget director of the other 
institution was consulted in order to make the determination of where the unit or position 
fit organizationally. 

Departments and positions in the other institutions were then rearranged, as 
necessary to match Northeastern s organizational structure as closely as possible. In some 
cases, this meant that two units from another institution were combined in order to be 
comparable to one unit at Northeastern. In other cases, a unit at another institution was 
split into two or more pieces in order to equal the organizational structure in place at 
Northeastern. In those departments in which positions were moved to other departments 
to make them comparable, no attempt was made to move an equivalent amount of non- 
personnel dollars from one department to another. To do so would have required better 
information on the relationship between positions and non-personnel expenditure levels 
than is currently available. 

This effort resulted in a crosswalk being prepared by vice presidential area which 
shows which departments and, in some cases, positions at the other institutions are 
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comparable to those at Northeastern. An example of what part of a crosswalk looks like 
is shown in Table 1. At the end of the crosswalk for each vice presidential area, a list of 
non-comparable units is shown. Examples of such units are central supply and duplicating 
services operations. At several other institutions the units are directly supported by state 
funds whereas at Northeastern these operations are funded on a cost recovery basis. Any 
non-comparable units were excluded from subsequent analysis. 

These crosswalks were then shared with the budget directors at the other institutions 
and with the vice presidents at Northeastern. The vice presidents were also encouraged 
to share this information with their unit managers so that any misclassifications could be 
corrected. All individuals were asked to review the crosswalks and comment on any 
omissions or misclassifications. The crosswalks were subsequently adjusted based on 
comments received. Based on the adjusted crosswalks, tables were prepared for each vice 
presidential area which show budgeted staffing levels and expenditures levels for each 
comparable unit for FY1987 and FY1988. Tables 2 and 3 show examples of what these 
tables look like. It should be noted that the staffmg level data show the total headcount 
for the unit, excluding temporary positions and student aides, regardless of the number of 
months for which a position has been budgeted. For departments where positions are 
being phased out, the headcount will be slightly overstated but this fact is reflected in lower 
costs in the budgeted expciiditure data and will be self-correcting in future years' analyses 
of staffing levels. 
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Table 1 

CROSSWALK BETWEEN COMPARABLE DEPARTMENTS 
AT PEER INSTITUTIONS 
FY1988 



PRESIDENT'S AREA 

Institution 
A 

President's Office (10100) 



Affirmative Action (10101) 



Department Name and Number 



Institution 
B 

Reimbursable Account (1 1 201 ) 
Office of the President (1 1200) 
except for A461 and H841 

Office of the President (1 1 200) 
A461 & H841 only 



Institution 
0 

President's Office (0255) 
Reserve-President's Area 
(0263) 

Affirmative Action Program 
(0259) 



Institution 
President's Office (11000) 



Affirmative Action (50100) 



ADMINISTRATIVE AFFAIRS 



Department Name and Number 



Institution 
A 

Administrative Affairs (20110) 
and (20990) 



Business Services (21120) 



Controller (21122) 



Institution 
B 

V.P. for Administrative Affairs 
(13300) 

Business Operations (13209) 



Controller (13212) except 
J316, H268, H723, H875 
Property Control (13219) 



Institution 
C 

Admin. & Finance V.P. (0271) 
Reserve-Admin. & Finance 
(0264) 

Treasurer's Office (0241) 



Accounting Office (0242) All 
except P970, P858, P859, 
P257, PI 74 

Property Control Operations 
(0245) 



Institution 
D 

V.P. for Business Affairs 
(12000) 



Business Services (51000) 
except for A003 

Accounting Office (51200) 
Business Sendees (51000) 
A003 only 
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Table 2 

COMPARISON OF BUDGETED EXPENDITURES IN NON-ACADEMIC 
DEPARTMENTS AT PEER INSTITUTIONS BASED ON 
FY1987 & FY1988 BUDGETS 



Vice Presidential Area 
President's Area 



Institution 
A 

1987 1988 



Institution 
B 

1987 1988 



Institution 
0 

1987 1988 



Institution 
D 

1987 1988 



President's Office 
Personal Sen^ices 
Support Costs 
Total 



20421 ZOO 
47163.00 
251375.00 



2O207Z00 
45939.00 
248011.00 



246888.00 
58198.00 
305086.00 



239090.00 
55270.00 
294360.00 



157327.00 
27215.00 
18454Z00 



168360.00 
20345.00 
188705.00 



160835.00 
23748.00 
184583.00 



209627.00 
21700.00 
231327.00 



Affirmative Action 
Personal Services 
Support Costs 
Total 



107993.00 
4950.00 
112943.00 



104724.00 
4950.00 
109674.00 



42784.00 

£S 

42748.00 



56748.00 

.OQ 

56748.00 



40381.00 
5860.00 
46241.00 



45020.00 
4356.00 
49376.00 



32271.00 
4823.00 
37094.00 



32562.00 
4282.00 
36844.00 



Grand Total - President's Area 

Personal Senses 312205.00 306796.00 289636.00 295838.00 197708.00 213380.00 193106.00 242189.00 

Support Costs 52113.00 50889.00 58198.00 55270.00 33075.00 24701.00 2 8571.00 25982.00 

Total 364318.00 357685.00 347834.00 351108.00 230783.00 238081.00 221677.00 268171.00 



Academic Affairs 

Provost's Office 
Personal Sen^ices 
Support Costs 
Total 



294972.00 
75720.00 
370692.00 



295044.00 
75720.00 
370764.00 



285919.00 
74972.00 
360891.00 



333421.00 
71536.00 
404957.00 



244001.00 
65366.00 
309367.00 



24092700 
50066.00 
290993.00 



322989.00 
24061.00 
347050.00 



344955.00 
18018.00 
362973.00 



Graduate College 
Personal Sen/ices 
Support Costs 
Total 



240024.00 
12419.00 
252443.00 



21884.00 
17025.00 
235869.00 



113382.00 
10620.00 
124002.00 



115297.00 
10032.00 
125329.00 



87717.00 
15600.00 
103317.00 



92683.00 
16132.00 
107815.00 



196751.00 
38581.00 
235332.00 



180783.00 
36045.00 
216828.00 
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Table 3 

COMPARISON OF vSTAFFING LEVELS (HEADCOUNT) IN NON-ACADEMIC 
DEPARTMENTS AT PEER INSTITUTIONS BASED ON 
FY1987 & FY1988 BUDGETS 



Vice Presidential Area 
Presidem's Area 



Institution 
A 

1987 1988 



Institution 
B 

1987 1988 



Instltutton 
C 

1987 1988 



Institution 
D 



1987 



1988 



President's Office 
A&P 

Civil Service 
Total 



2.00 
3.40 
5.40 



2.00 
4.40 
6.40 



3.00 
4.00 
7.00 



3.00 
4.00 
7.00 



2.00 
2.00 
4.00 



3.00 
2.00 
5.00 



1.50 
3.00 
4.50 



1.50 
3.00 
4.50 



Affirmative Action 
A&P 

Civil Sen^ 
Total 



2.00 
1.00 
3.00 



2.00 
1.00 
3.00 



1.00 
1.00 
2.00 



1.00 
1.00 
2.00 



1.00 
1.00 
2.00 



1.00 
1.00 
2.00 



.50 
1.00 
1.50 



.50 
1.00 
1.50 



Grand Total 
A&P 
CivH Service 
Total 



President's Area 
4.00 



4.40 
8.40 



4.00 
5.40 
9.40 



4.00 
5.00 
9.00 



4.00 
5.00 
9.00 



3.00 
3.00 
6.00 



4.00 
3.00 
7.00 



2.00 
4.00 
6.00 



2.00 
4.00 
6.00 



Academic Affairs 

Provost's Office 
A&P 

Civil Service 
Total 



4.00 
4.00 
8.00 



4.00 
4.00 
8.00 



4.00 
5.00 
9.00 



4.00 
4.00 
8.00 



5.00 
4.00 
9.00 



6.00 
3.00 
9.00 



5.00 
4.50 
9.50 



5.50 
4.50 

10.00 



Graduate College 
A&P 

Civil Sen^ 
Total 



4.00 
3.00 
7.00 



3.50 
3.00 
6.50 



1.00 
3.00 
4.00 



1.00 
3.00 
4.00 



2.00 
4.00 



2.00 
1.50 
3.50 



3.00 
4.50 
7.50 



3.50 
4.50 
8.00 
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Unit of Analysis. While the type of detailed information provided in 
Tables 2 and 3 was ultimately provioed to the vice presidents and unit managers as 
backup, it was never intended to be used as a basis for evaluating the relative efGciency 
of nonacademic units. As noted in the previous subsection, a great deal of time and effort 
was expended in making the units as comparable as possible. However, in spite of these 
efforts, it was recognized that subtle nuances exist and that not all of the functions 
performed by similar departments are the same across institutions. For example, while the 
Budget Office at Northeastern is responsible for the expenditure oversight and position 
«.wntrol functions, these functions are performed by the Accounting OfiGce at another 
institution. While in theory it would have been possible to analyze the portion of time 
each position spent on these functions and reassign the percentage of the positions to the 
Budget Office, in practice the costs involved in developing such detailed information to 
achieve comparability greatly outweigh the benefits to be derived from such practices. For 
this reason, the advice of Dunn (1987) and others was heeded and the highest useful level 
of aggregation of data possible was used for the unit of analysis. 

In order to determine what the appropriate unit of analysis might be, the primaiy 
functions of each unit were analyzed and units grouped together based on similarity of 
functions. Only a handful of units were grouped into functional areas based on this 
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analysis. Logical groupings developed included physical plant operations, library and media 
services, business services, budget, planning and institutional research, and career and 
personal counseling. Comparative data were provided for these functions. With these 
exceptions, the vice presidential level was determined to be the most appropriate unit of 
analysis for the evaluation of the relative efGcienc^ of nonacademic units. 

Lmvml of Analysis. Based on the d ta already generated, the simplest 
means of determining the relative efficiency of nonacademic units would be to compute 
a unit cost per staff for each vice presidential area by dividing total budgeted expenditures 
by total staffing levels. However, this methodology was not used to determine relative 
efGciency for several reasons. First, the unit costs factor makes no allowances for 
differences in the cost of living between the Chicago metropolitan area and downstate 
Illinois. Since salaries for support positions, in particular, are sensitive to market 
conditions, the use of a unit cost factor to determine the relative efficiency of nonacademic 
units may not produce comparable results. Secondly, the unit cost factor may also be 
skewed by the extent to which differences in the length of employee service affects total 
salaries. 

Therefore, a different measure was sought, one which would be less sensitive to 
price and other extraneous factors. Two measures were ultimately chosen: fall headcount 
enrollment and fall full-time equivalent (FTE) enrollment. While most higher educaiion 
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data is measured on an FTE basis, it was decided to also measure nonacalemic units on 
a headcount basis given that some of the units such as the Admissions and Cashiers Office 
provide the same level of service to each student regardless of the number of student 
credit hours taken. 

Tables 4 and 5 provide a comparison of budgeted expenditures per headcount/FTE 
student and of headcount/FTE. students per budgeted staff. The use of four separate 
indicators helps address the concern that the data may not be perfect. As Ewell (198S) 
has indicated, if multiple indicators are all pointing in the same direction, even when 
derived from crude data, then something authentic is usually being captured. 

It then becomes the responsibility of unit man^^ers and executive management to 
interpret what is being captured. For example, if one institution consistently shows up as 
budgeting fewer staff in nonacademic units on either a headcount or student basis, does 
this mean that their employees are more productive than those at the institutions or that 
the output expected from the unit is less than at the other institutions? Do technological 
differences between offices account for any of the variance? To what extent is a low level 
of expenditure for nonacademic units indicative of the institutional priority assigned to 
instruction, research and^o^ public service? In summary, the comparative data will have 
served their purpose well if they lead to the tyf • . of questions raised ab'^ve Ideally, the 
information should enable executive decision-makers to determine wh^.ner tne level of 
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Tabl6 4 

COMPARISON OF BUDGETED EXPENDITURES PER HEADCOUNT/FTE STUDENT * 
IN SEI^CTED NON-ACADEMIC DEPARTMENTS FOR FY1987 AND FY1988 



VP Ara«/Function 


Irwl 

IIRH 

A 


1987 
Headcount 
Inst Inst 
B C 


Inst 
D 


Inst 
A 


1988 
Hsadcount 
Inst Inst 
B C 


Inst 
D 


Inst 
A 


Inst 
B 


1987 

FTE 
Inst 
C 


Inst 
D 


Inst 
A 


Inst 
B 


1988 

FTE 

Inst 
C 


Inst 
D 


Pmkient's Area 


34 


45 


22 


18 


34 


50 


22 


21 


57 


72 


23 


22 


57 


81 


24 


25 


AdminMratlve Affairs 

Amb PhvslMil Pljinf\ 
rnyaivoi rtOSn) 


188 


232 


164 


163 


186 


263 


154 


156 


316 


373 


178 


195 


315 


427 


167 


188 


Physical Plant Opsrations 


294 


417 


335 






465 


351 


394 


493 


670 


362 


510 


534 


755 


382 


474 


Administrative Affairs 
(Inci. Physical Plant) 


483 






589 


502 


728 


505 


551 


809 


1043 


540 


705 


849 


1182 


549 


662 


Student Affairs 


129 


140 


119 


139 


127 


171 


126 


120 


216 


225 


129 


166 


215 


278 


137 


144 


Development & Fubito Affairs 


63 


76 


34 


92 


80 


79 


36 


74 


105 


122 


36 


110 


135 


128 


39 


89 


Soledod FurK!tions* 


































Business Services 


90 


101 


54 


81 


90 


102 


56 


78 


ISO 


163 


59 


98 


152 


165 


61 


94 


Budget & institutional 
Studies and Planning 


30 


38 


20 


20 


28 


41 


20 


19 


SO 


60 


22 


24 


47 


66 


21 


23 


Library & University Media 
Services 


266 


214 


230 


259 


242 


233 


225 


220 


445 


344 


249 


310 


409 


377 


245 


264 


Counseling and Career 
Planning and Placement 


56 


69 


43 


44 


56 


73 


48 


48 


94 


110 


47 


52 


94 


119 


52 


58 



28 29 

o 

ERIC 



Table 5 

COMrARISONflOF HEADCOUNT/FTE STUDENTS PER BUDGETED STAFF 
IN SELECTED NON-ACADEMIC DEPARTMENTS FOR FY1987 AND FY1988 



1987 
Headcount 



1988 
Headcount 



1967 
FTE 



1968 
FTE 



VP Ared/Function 


Inst 
A 


Inst 
B 


lilSt 

c 


Inst 

n 


Inst 

A 
M 


Inst 

R 


Inst 

w 


Inst 

U 


Inst 

A 
M 


Inst 

R 
D 


Inst 

w 


Inst 
u 


Inst 

A 
M 


Inst 

R 
D 


Inst 

p 


Inst 

U 






ooc.o 


1 r 0.9 


OA19 K 


1125.3 


769.6 


1537.3 


2116.3 


799.7 


537.2 


1640.8 


1681 .7 


669.3 


4o4.1 


141 2.9 


1760.9 


(ItM Phytical Plant) 






1KA a 
190.0 




111 <> 
191 .9 


93.1 


162.0 


194.4 


79.9 


64.5 


144.9 


121 .3 


77.6 


K7 4 

97.3 


148.9 


4AA A 

128.4 








Q1 1 


A7 A 
Of .O 


67.2 


68.0 


90.4 


71.9 


97.1 


49.2 


84.1 


56.5 


K4 A 

91 .6 


AA A 

41 .9 


A1 4 

63.1 


KA K 

99.9 


Adminittmtiva Affairs 
ffnel Phvftieal PlAnt) 


54.6 


42.7 


57.6 


46.1 


52.4 


39.3 


58.0 


48.9 


32.6 


26.6 


53.2 


38.6 


31.0 


24.2 


53.3 


40.7 


Studant Affairs 


212.1 


191.7 


202.2 


192.7 


225.5 


170.4 


200.5 


212.5 


126.6 


119.4 


186.9 


161.1 


133.3 


105.0 


184.2 


176.8 


Davalopmant & Public Affairs 


470.7 


388.2 


968.5 


30?.2 


-ra.i 


392.9 


1092.5 


342.5 


280.9 


241.8 


895.0 


253.4 


220.6 


242.1 


1004.1 


284.9 


Salactad Functions: 


































Businass 8anHce$ 


2S9.5 


221.3 


355.1 


268.9 


266.1 


228.1 


371.1 


279.4 


154.8 


138.1 


3212 


224.7 


157.3 


140.5 


341.0 


232.4 


Budgat & Inatitutionai 
Studlaa and Phnning 


967.1 


776.3 


1331.6 


1509.4 


1057.8 


642.9 


1345.1 


1587.3 


577.1 


483.5 


1230.6 


1261.3 


625.4 


396.1 


1236.3 


1320.4 


Ubrary & Unlvarslty Madia 
Sarvlcas 


126.0 


137.4 


156.5 


150.5 


132.1 


125.2 


156.7 


161.2 


75.2 


85.6 


144.6 


125.7 


78.1 


77.1 


144.0 


134.1 


Counsaling and Caraar 
Planning and Placamant 


480.3 


369.7 


608.7 


647.5 


525.0 


353.6 


581.7 


612.0 


286.6 


230.2 


562.6 


541.0 


310.4 


217.9 


534.6 


500.1 
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resources being provided is optimal relative to overall institutional priorities and expected 
output. 

Information Souroos* Given the level of detailed information required to 
make the nonacademic units as comparable as possible, either budget or expenditure data 
are needed to conduct the analysis. While budget data were used in the comparisons, 
similar expenditure data are usually available from internal financial records and financial 
reports. Many of the comparisons discussed previously were first tested using both budget 
and expenditure data. The budget data proved to be a more reliable indicator over time 
than the expenditure data since the expenditure trend data would reflect artificial decreases 
in expenditures between years whenever positions remained vacant for any length of time. 
Since the budget data are based on unit plans, unexpected vacancies do not present a 
problem. 

Perhaps the best information source proved to be those unit managers who took the 
evaluation process seriously and informed the researcher when units or positions were 
being misclassified between institutions. With the crosswalk of units and positions serving 
as the foundation of the comparative analysis, the feedback received from unit mangers 
helped strengthen the results. 

Pitfalls to Avoid. Since detailed information is collected and analyzed 
using this framework, consideration must be given to at what point the cost of collecting 
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and analyzing the information exceeds the potential benefits to be derived. The size and 
complexity of the institution would likely dictate the decision. For example, this 
methodology would be extremely difficult to implement for anything other than selected 
units at a major research university. For comprehensive universities, while it might be 
tempting to collect similar rformation from peer institutions in other states, the 
cost/benefit of such a venture would need to be weighted carefully. 

In a comparative analysis such as this, it is extremely difficult for unit managers not 
to feel threatened by the process. Thei" participation in selecting comparable units at 
other institutions should alleviate some of these concerns. However, it is just as likely that 
once the data are analyzed, concern will be expressed that the units are not comparable 
due to the use of different technologies or processes to accomplish the same function. If 
the comparative data lead unit managers to make statements like this, then it will have 
served at least part of its purpose by calling attention to the fact that similar functions can 
be accomplished with different levels of resources and expectations. The use of multiple 
indicators to verify that each indicator is saying the same thing will also alleviate some of 
unit managers' fears. 

Finally, on a more technical note, when analyzing the financial information, care 
must be exercised in order to avoid such data traps as whether certain costs are handled 
on a chargeback basis or are centralized; e.g. administrative computing. While the cost 
differential may be inconsequential for most items, it could greatly skew the results for 
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some units. For example, if administrative computing costs are charged to the accounting 
office in one university but not the other, the cost differential between the two units would 
be artificially high. 

Postscript. While the nonacademic program review process instituted at 
Northeastern in 1982 was subsequently abandoned, comparative staffing and budget data 
are still being collected periodically on nonacademic units. In fact, some vice presidents 
now raise questions about how units' staffing levels compare to their peers. The 
acceptance of the fi-amework became very clear in 1989 vhen several ;ice presidents 
utilized the information in determining where staffing levels needed to be reduced in order 
to respond to a sizeable reduction in the state fund budget This brought home the point 
made by Patton in which he stated that "utilization occurs when there is an immediate, 
co-icrete and observable effect on specific decisions and program activities resulting directly 
fi-om evaluation findings" (Wergin and Braskamp, 1987:94). 
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